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This paper is in the nature of an interim report on an evaluation 
exercise which has run half its course. It attempts to outline 
the characteristics of the curriculum intervention which is 
being evaluated, the impact of that intervention upon the 
educational system, and the attempt to design an appropriate 
evaluation. If this account of the evaluation is sometimes 
cryptic and impressionistic, I hope it is because I have tried 
to convey in a limited time both the nature of its empirical 
roots and the span of its concerns. 
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The Humanities Curriculum Research and Development Project was 
set up in 1967 as part of the preparation for the raising of the 
school-leaving age, which, in 1972 when the Project ends, will 
be raised from 15 to 16 years of age for all students. The 
.Project is jointly financed by the Nuffield Poundation, an 
independent trust fund, and the Schools Council, a committee, 
mainly of teachers, administering research funds provided partly 
from national government sources ana partly from local government 
education budgets. The evaluation unit, which is accountable to 
the Project Director, has a four-year life from 1968 to 1972, 
and is financed solely by the Schools Council. The total budget 
for the Project and the evaluation is approximately a quarter 
of a million pounds. 

The central team was asked to provide stimulus, support and 
materials for teachers and schools teaching the humanities to 
adolescent students aged- 14- to 16. In order to give focus to 
its research, the central team defined humanities as the study 
of important human issues. They decided to concentrate on the 
special problems of work in controversial areas and to give 
support and produce materials to enable the teacher to meet 
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these problems. Following up policy statements made by 
educationalists and official reports, they saw humanities 
teaching as a possible response to the demand that the curriculum 
offered to adolescents should be relevant and that schools should 
face controversial issues with these students in an honest and 
adult way. The problem, as they saw it, was how to allow 
adolescents to reach views responsibly without being restricted 
by the teacher's bias or subjected to undue pressures by their 
fellows. They approached this problem by attempting to stimulate 
and study a pattern of small group discussion in which collections 
of evidence, printed prose, verse, drama, photographs, paintings 
and tapes, were used in a discussion situation which placed both 
teachers and pupils under the discipline of evidence. They 
produced collections of materials in such areas as war, education, 
relations between the sexes, the family, poverty, people and work, 
race, living in cities and law and order. The role of the central 
team has been to put forv/ard initial hypotheses about teaching 
strategies in a discussion situation where the teacher acts as a 
neutral chairman and a resource consultant. During the years 
1968 to 1970, the first collections were used experimentally in 
thirty-six schools throughout England and Wales. Teachers in 
those schools were asked to test the suggested rules for discussion 
to help to shape the work in research and in the creative 
activities that needed to be built' up around the discussion, which 
was envisaged as a core activity in an open-ended enquiry 
curriculum. 



From Easter 1970 , revised packs began to be published commercially 
and training schemes for teachers were set up throughout the 
country to meet the response from individual schools and local 
education authorities. During the current academic year some 
four to five hundred schools are using the materials, which are 
available on the open market. 
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In August 1968, I was hired by the Schools Council to study this 
human issues project during its two-year trial period and to design 
an evaluation of it for implementation in 1970 to 1972 when the 
Project would be in nationwide dissemination. The Project 
Director had asked for an independent evaluation and I was given 
a free hand. During the past year I have been joined by three 
colleagues and the four of us are currently engaged in carrying 
out the evaluation programme. 



/In conceptualisation the Project 



In conceptualisation the Project had three striking characteristics: 

1. It was attempting to operationalise an attitude towards 
curriculum which stressed teacher responsibility and judgement, as 
against an attitude which sees the development team as educational 
leaders who dispense expertise from a position of authority. By 
seeking at most a relationship of equality with participating 
system personnel, and emphasising the research role of those 
involved, the central tenia invited teachers to become technologists 
of curriculum rather than technicians. 

2. In adopting 'understanding' as the aim of the programme the 
central team expressed a faith in educational rather than social 
adjustment approaches to controversial issues, a key problem in 
the development of a secondary school tradition in Britain. Some 
core values embodied as research hypotheses in the programme 
seemed dissonant with established school practice, particularly 
with respect co teacher and pupil roles. 

5« The theoretical design of the Project started with a 
specification of content, followed by the formulation of an aim. 

The aim was then analysed into principles of i^rocedure or process 
criteria, and an experiment mounted to try to realise in the 
classroom a pedagogically effective 'form' which would embody 
these criteria. It was therefore highly dependent on the teacher's 
understanding of philosophic and pedagogic principles for intelli- 
gent classroom practice. 



Although the central team's general position in these areas was 
reasonably clear at the design stage, it was not until its 
policies became operationalised in the school system during the 
1968 to 1970 experiment that the implications and consequences 
could be appreciated, and the related technical problems articulated 



I was appointed on the eve of the experiment to evaluate an 
innovation which, at first glance, bore many of the earmarks of 
past failures in the field. It required induction courses for 
teachers, it was difficult to use, it was costly in terms of 
school resources, it conflicted with established values. In short, 
the Project showed distinct promise as a case-study in the 
pathology of innovation, from symptoms to post-mortem. 



Such a perspective did not survive its agronomic assumptions about 
criteria of success and failure. It soon became obvious that to 
base such criteria exclusively or even predominantly on short- 
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term pupil learnings is inappropriate where the programme under 
consideration constitutes a radical intervention in the whole 
organisational structure of a school system. 

At that early stage of weighing up possible approaches to 
evaluation, three considerations seemed paramount. 

1. The sponsor of the evaluation was a government agency with 
responsibility for national curriculum development , but with 
little exper-ienco in this role. There was therefore a need for 
information that would aid planning at this level. This suggested 
that one focus for evaluation might be upon the patterns of inter- 
action within the system initiated or illuminated by project 
inputs. 

2. Because the central team decided to shore neci si on-making with 
participating schools, and because the mediational influence of 
different educational milieux seemed likely to prove to be a 
significant variable, considerable divergence in institutional 
response was anticipated. Field studies of the Project in a 
variety of operational contexts would be called for. 

3. The climate of educational theory in Britain strongly 
favoured the behavioural objectives model of curriculum develop- 
ment and evaluation. Little thought had been given to alternative 
models, and I was quite unaware then of recent American debate 

in this area. My aim was simply to describe the work of the 
Project in a form which would make it accessible to public and 
professional judgement. Evaluation design, strategies and 
tactics would, hopefully, evolve in response to the impact of 
the Project on the- system and the structure of the evaluation, 
problems which that impact would throw up. 

The Project in the Experimental Schools: 1968/70 

The 36' schools which mounted the experiment in the autumn of 1968 
xvere not selected by sampling methods. They were nominated by 
their administering authorities, and reflected by their variety 
interesting differences in judgement and priorities among the 
local education authorities. The participating teachers got 
together with the central team during the summer at regional 
conferences where the nature and design of the experiment was 
explained to them and their task outlined. By all accounts most 
of them went away from these conferences with some enthusiasm 
for the task. , . 

/These, experimental schools 
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These experimental schools embraced a wide range of environmental, 
compositional, structural and cultural variables. This contextual- 
diversity was compounded by differences in the decisions they made 
about how to introduce- , organise and implement the experiment , and 
further complicated by differences among participating personnel 
of motivation, commitment, understanding and expectations. Yet 
another variable was the extent and nature of support of the 
local authority. 

The immediate impact of the Project v:as on. the whole alarming. 

There was enormous confusion and misunderstanding, leading to a 
general failure to respond appropriately. There we re many 
unanticipated problems and widespread misperception of the demands 
that the Project was making. Some elements in this were: 

1. The importance of headmasters in innovation w as under- 
estimated by the central team, particularly in view of the 
conditions which were necessary for the implementation of the 
experiment, the support structure which such difficult and novel 
work seemed to call for, and its implications for the school's 
authority structure. The Project was seen to be manipulating 
major variables in the school, including established patterns of 
social control, a concern very real to those who work in schools. 
The Project did not see initially the demands it was making on 
rather inflexible administrative institutions. It was not easy 
for schools to create the necessary conditions for the experiment. 

2. The teachers did not anticipate the extent to which many 
students had developed a trained incapacity for this work , nor the 
depth of alienation from any kind of curriculum offering which 
many students felt, nor the degree to which they themselves and 
their students bad“ been successfully socialised into a tradition of 
teacher dominance and custodial attitudes. Many became locked in 
role conflicts, others in attempts to bridge an unforeseen 
credibility gap between themselves and their students. 

5. It emerged that the central team had failed at the outset to 
communicate successfully the nature of the enterprise. Prom the 
teacher's point of view the ethos of the Project was evangelical 
rather than exploratory, and too suggested teaching strategies 
tests of teacher proficiency rather than research hypotheses. 

Many felt on trial. This both reduced their capicity to profit 
from the experience and adversely affected their feedback to the 
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Had the picture that emerged from the schools been as uniform as 
those- points might suggest, perhaps the evaluation would have 
developed differently. But it was not. Although the programme 
proved generally to be demanding, difficult and disturbing, there 
were striking exceptions and many contradictions in reported or' 
observed phenomena. 'Limited' explanations of perceived failure/ 
success, such as pupil ability, teacher behaviour, or institutional 
ethos, could not be rc-adily generalised. The matter seemed 
increasingly complex. • 

During the first year, while the central team grappled with the 
problems of the- schools in an effort to operationalise the 
experiment in a fora which would allow them to relate pupil 
response to controlled teacher behaviour, the evaluator concen- 
trated on trying to establish precisely what was happening in the 
schools, and on gathering information that might help to explain 
differing patterns of action and response. I studied the 
behaviour of the central team and the- interaction between them, 
the local authorities, and the schools, gathered data for each 
school about the external forces of support and opposition that 
were mobilised by the Project's intervention, got out a checklist 
of hard and soft data items which added up to an institutional 
profile of each school, tried to assess, by questionnaires 
administered at conferences, the participating teachers' 
understanding o‘f Project theory and attitudes towards it, and got 
the teachers "to send in audiotapes of their classroom discussions 
together with written supplementary data. The needs of the central 
team and of the evaluation overlapped sufficiently to form a 
continuing basis of cooperation, even if the demands of their 
support role made it increasingly difficult to match priorities. 

I embarked on a series of visits to the schools, intending 
initially to study all of them at first hand. After visiting about 
half of them this plan was abandoned in favour of case- studying a 
small number, principally because I could not understand the 
causes of the behaviour which I had observed in discussion groups. 
Why were the. differences between • schools in this respect so much 
greater than the differences within schools? Why was one group of 
pupils enthusiastic about the work, and a similar group in another 
school so hostile? Other questions accumulated as one began to 
seek contextual clues. Why were some staff groups supportive of 
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the Project, others indifferent, still others openly hostile? 

Why did some schools x'oact in dissimilar ways to apparently 
similar problems? A host of Questions like these aro/e as the 
diversity of institutional, teacher and pupil response unfolded. 

Towards the end of the first year, and throughout the second year 
of the experiment, eight schools wore studiod closely. It is 
not possible here to give an account of those studios, but I can 
list some of the propositions arising out of them that the 
evaluation intends to explore. 

1. Human action in educational institutions differs widely 
because of the number of variable influences that determine it. 
This is obvicus, yet in curriculum evaluation it is sometimes 
assumed that what w as intended to happen is what actually happens 
and that v/hat happens varies little from setting to sotting, 

2. The impact of an innovation is not a set of discrete effects, 
but an organically related, pattern of acts and consequences. 

To understand fully a single act one must locate it functionally 
within that pattern. It follows from this proposition that 
curriculum interventions have many more unanticipated consequences 
than is normally assumed in development and evaluation designs. 

3° No two schools arc sufficiently alike in their circumstances 
that prescriptions of curricular action can adequately supplant 
the judgement of the people in them. Historical/evolutionary 
differences alone made the innovation 'gap' a variable which has 
significance for decision-making. 

4-. The goals and purposes of the programme developers are not 
necessarily shared by its users. We have seen the Project used 
variously as a political resource in an existing power sti'uggle, 
as a way of increasing the effectiveness of a custodial pattern 
of pupil control, and as a means of garnishing the imago of 
institutions which covet the v/rappings, but not the merchandise, 
of innovation. 

Further considerations in the development of the evaluation 
design 

1. Audiences 

Faced with a central team who were opposed, to the use, of 
‘objectives', I had to look elsewhere for a concept of 
evaluation to guide me. In any case, as I became aware of the 

of what was going on in the. 
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experimental schools, I he came increasingly sceptical of the 
notion of confining evaluation to the measurement of intention 
achievement. 

I then explored the possibility of defining my responsibilities 
in relation to likely readers of my report. The idea of 
evaluation for consumers attracted mo. In tine 'consumers' 
became redefined -as decision-makers and four main groups of 
dc-cision-makers emerged - the sponsors, the' local education 
authority, the schools, and the examination boards'. The task of 
evaluation was then defined as that of answering the questions 
that decision-makers ask. 



This task definition was subsequently perceived ns unsatisfactory 
principally because it assumed that these pc-ople knew in advance 
what questions were appropriate. At the present moment we see our 
task as that of feeding the judgement of decision-makers by 
promoting understanding of the considerations that bear upon 
curricular action. Our orientation here is towards educing an 
empirical rather than a normative model of educational decision- 
making and its consequences. 

2. 'Data 







Decision-making groups differ in their data requirements. 

Individuals differ in the. • degree of confidence they place in 
different kinds of data, and in the levels of confidence at which 
they are prepared to act. The evaluation is taking account o.f 
this by trying to integrate both subjective and objective approaches 
(to use a convenient if misleading dichotomy) , in a very broad 
study of the Project from 1970 to 1972. 

3 . Pocus 

Much • evaluation work in the past has been simplistic in its 
assumptions., or so subservient to canons of experimental 
psychology that its attention has been too narroi\rly focused. 
Education is a complex practical activity. Any effort to reduce 
that complexity to singularistic perspectives tends to distort 
the reality, and may mislead those who seek to understand the 
reality. Least of all does it help those - who live there. 

Perhaps at this stage of our under standi n- bolder evaluation 
: designs can give us some- more adequate view : of wliat it is we are • 

trying to change, and of what is involved in: changing it. It 
^ is this belief that lies behind a wholistic approach to evaluation. 
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A systematic attempt will bo nr.de to document the outcomes of 
the Project at the levels of system, institution, teacher end 
pupil, to locate those outcomes within a record of antoecdnnt 
events and conditions, and to explain variations in out come in 
terms which will lead to. mere informed curriculum decisions. 

This is an ambitious aim in such a complex area and represents 
for us an aspiration rather than an achieveablc goal. 

Evaluation Design ■ 

a) In a largo sample of schools (c. 100) 

(i) Gathering input, contextual and implementation data by 
questionnaire. 

(ii) Gathering experiential impressionistic and .judgemental 
data from teachers and pupils. 

(iii) Objective measurement of pupil and teacher change. (We 
have at the beginning of this year carried, out -pre- 
tests of pupils on 21 objective instruments which 
represent the combined judgements of teachers, pupils, 
the central team and ourselvc-s, of likely dimensions of 
pupil change). 

(iv) Tracing process variation by multiple choice feedback 

instruments which require minimal effort by the teacher 
and are monitored by pupils. 

(v) Monitoring institutional response by semi- structured 
diary instruments. 

b) In a small sample of schools (c.12) 

(i) Case studios of patterns of decision-making,; 

communication, training and support in local areas. 

(ii) Case studies of individual schools within, these areas, 

(iii) Study of process dynamics by audiotape, videotape, and 
observation. 

To sum up, we now have the job of describing the experience of 
hundreds of schools embarking on work with Project materials, and, 
moreover, describing that experience in ways which, will be helpful 
to those who 'have. to make judgements in this field. In an open- 
ended enquiry programme of this kind .where- the aims, are broadly- 
defined and where teacher behaviour is a matter of individual 
response to particular situations in the light of these aims, this 

/is no simple matter. 
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is no simple matter. We 
studies and quantitative 
to bo combined in a broad 



see the need for both 
data gathering and mo a 
front approach to the 



qualitative field 
sure moot techniques 
evaluation. 



It is hoped, by interweaving these studies, 
understanding ox the interplay of forces in- 



to advance 
this curriculum 



innovation. 
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